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DETAILED ACTION 

Continued Examination Under 37 CFR 1.114 

1 . A request for continued examination under 37 CFR 1.114, including the fee set 
forth in 37 CFR 1 .17(e), was filed in this application after final rejection. Since this 
application is eligible for continued examination under 37 CFR 1.114, and the fee set 
forth in 37 CFR 1 .17(e) has been timely paid, the finality of the previous Office action 
has been withdrawn pursuant to 37 CFR 1.114. Applicant's submission filed on 
04/05/2010 has been entered. 

Remarks 

2. Receipt of Applicant's Amendment, filed on 04/05/201 0, is acknowledged. The 
amendment includes the amending of claims 1, 4-9, and 19, and the cancellation of 
claims 2-3, and 18. 

Priority 

3. Receipt is acknowledged of papers submitted under 35 U.S.C. 1 1 9(a)-(d), which 
papers have been placed of record in the file. 

Claim Rejections - 35 USC § 103 

4. The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

5. Claims 1,3-19 are rejected under 35 U.S.C. 1 03(a) as being unpatentable over 
Hyon (U.S. PGPUB 2002/0077135) in view of Baker (U.S. Patent 6,546,417), and 
further in view of Ostermann et al. (U.S. Patent 6,990,452). 

6. Regarding claim 1 , Hyon teaches a display device comprising: 

A) a first storage portion for storing beforehand a character code for specifying each of 
said predetermined kind of characte r, and character shape data corresponding to said 
character code in a correlated manner (Paragraphs 23, and 41-42, Figure 4); 
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B) a second storage portion for storing an image code for specifying said registered 
image and non-standardized image data corresponding to said image code in a 
correlated manner according to registration processing by a user (Paragraphs 23, 25, 
and 44, Figure 2); 

C) a display output portion for outputting said predetermined kind of character and said 
registered image (Paragraph 21, Figure 2); and 

D) a display control portion for causing said display output portion to output 
corresponding said predetermined kind of character and said registered image both 
belonging to an identical sentence based on display data containing a series of said 
character code, text attribute data, and said image code (Paragraphs 21 and 41-42). 

The examiner notes that Hyon teaches "a first storage portion for storing 
beforehand a character code for specifying each of said predetermined kind of 
character , and character shape data corresponding to said character code in a 
correlated manner " as "A storage 18 is comprised of a ROM (Read Only Memory) and 
a RAM (Random Access Memory) for storing programs and data, and a voice memory. 
The storage 18 stores an operation program for inputting emoticons and a plurality of 
emoticons in the form of a bit map according to the embodiment of the present 
invention. The plurality of emoticons are stored by groups in the storage 18 to facilitate 
selection of emoticons" (Paragraph 23) and "the user can input a text including typical 
characters, special characters, or emoticons within the range of a transmittable SMS 
message, by changing input modes" (Paragraph 41 ). The examiner further notes that 
Hyon teaches "a second storage portion for storing an image code for specifying 
said registered image and non-standardized image data corresponding to said 
image code in a correlated manner according to registration processing bv a 
user " as "A storage 18 is comprised of a ROM (Read Only Memory) and a RAM 
(Random Access Memory) for storing programs and data, and a voice memory. The 
storage 18 stores an operation program for inputting emoticons and a plurality of 
emoticons in the form of a bit map according to the embodiment of the present 
invention. The plurality of emoticons are stored by groups in the storage 18 to facilitate 
selection of emoticons" (Paragraph 23), "storage 18 stores oriental or occidental 
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emoticons according to the cultural area of a user" (Paragraph 25), and "A plurality of 
emoticons are pre-stored in a mobile terminal so that a user easily select an intended 
emoticon in the present invention. In another embodiment, the user can add 
hieroglyphics to basic emoticons stored by a manufacturer. The hieroglyphics can be 
stored in three ways: the user directly stores them, receives them from a base station 
and the stores them, or downloads them from the Internet. The user can change and 
edit the emoticons stored in the mobile terminal" (Paragraph 44). The examiner further 
notes that it is clear that the emoticons which are edited, changed, and user-generated 
must be stored in the RAM of storage 18 since items stored in RAM can be written to 
whereas items stored in ROM cannot be written (i.e. are Read-Only). The examiner 
further notes that Hyon teaches "a display output portion for outputting said 
predetermined kind of character and said registered image" as "a display 16 
outputs display data and text messages generated in the mobile terminal. An LCD 
(Liquid Crystal Display) can be used as the display 16" (Paragraph 21). The examiner 
further notes that Hyon teaches "a display control portion for causing said display 
output portion to output corresponding said predetermined kind of character and 
said registered image both belonging to an identical sentence based on display 
data containing a series of said character code, text attribute data, and said 
image code" as "a display 16 outputs display data and text messages generated in the 
mobile terminal. An LCD (Liquid Crystal Display) can be used as the display 16" 
(Paragraph 21), "If the user selects "smile" in the display 6 by manipulating a left/right 
directional key, emoticons that belong to the group "smile" are displayed in a display 7. 
When the user chooses one of the "smile" emoticons " . " in the display 7, the selected 
emoticon is stored as part of the short message as shown in a display 8. Then, the user 
can input a text including typical characters, special characters, or emoticons within the 
range of a transmittable SMS message, by changing input modes" (Paragraph 41), and 
"After an intended text is completed, the user instructs the mobile terminal to transmit 
the SMS message including the emoticon... the mobile terminal transmits the stored 
message" (Paragraph 42). The examiner further notes that it is common knowledge 
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that when text messages are received via a device, the entire message is displayed at 
once. 

Hyon does not explicitly teach: 

F) wherein said display control portion determine whether or not a size of characters 
constituting the text where the registered image is inserted and a size of the registered 
image are substantially the same : 

G) when the size of characters constituting the text where the registered image is 
inserted and the size of the registered image are not substantially the same, said 
display control portion makes the size of the characters and the size of the registered 
image substantially the same by scaling up/down the size of the registered image based 
on the text attribute data : and 

I) said display control portion causes said display output portion to display a text 
containing characters and the registered image scaled to substantially the same size . 

Baker, however, teaches " wherein said display control portion determine 
whether or not a size of characters constituting the text where the registered 
image is inserted and a size of the registered image are substantially the same " 
as "in order to accommodate the use of different size fonts in the mailbox display, 
means for scaling the size of the icon graphics are also provided... at least one image 
for each icon is stored, the icon most closely matching the point size of the font is 
chosen and then scaled as needed to better match the font point size" (Column 8, lines 
26-40), " when the size of characters constituting the text where the registered 
image is inserted and the size of the registered image are not substantially the 
same, said display control portion makes the size of the characters and the size 
of the registered image substantially the same bv scaling up/down the size of the 
registered image based on the text attribute data " as "in order to accommodate the 
use of different size fonts in the mailbox display, means for scaling the size of the icon 
graphics are also provided... at least one image for each icon is stored, the icon most 
closely matching the point size of the font is chosen and then scaled as needed to 
better match the font point size" (Column 8, lines 26-40), and " said display control 
portion causes said display output portion to display a text containing characters 
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and the registered image scaled to substantially the same size " as "in order to 
accommodate the use of different size fonts in the mailbox display, means for scaling 
the size of the icon graphics are also provided... at least one image for each icon is 
stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Baker's would have allowed Hyon's to provide a method for allowing for scalable icons 
to accompany texts with specified and varied fonts in order to easily associate an icon 
to a particular sentence, as noted by Baker (Column 5, lines 40-55). 

Hyon and Baker do not explicitly teach: 
E) said text attribute data containing color attribute data including at least a fore color of 
a corresponding predetermined kind of character : 

H) said display control portion converts a color of said registered image according to 
said color attribute data . 

Ostermann, however, teaches " said text attribute data containing color 
attribute data including at least a fore color of a corresponding predetermined 
kind of character " as "The increased intensity of the emoticon may be accomplished 
by changing the icon from black-on-white background to black-on-colored background 
(such as red or yellow) where the intensity of the background color reflects the 
amplitude" (Column 9, lines 43-67) and "The sender can associate typed words with an 
emoticon by underlining, coloring, highlighting, or by any other means. For example, 
the method may comprise providing the sender an option to assign a color to the at 
least typed one word such that the chosen emoticon begins to be presented by the 
animated entity to the recipient at the first typed word with the assigned color and the 
chosen emoticon presentation by the animated entity ends at the last typed word with 
the assigned color" (Column 1 1 , lines 43-50) and " said display control portion 
converts a color of said registered image according to said color attribute data " 
as "The sender can associate typed words with an emoticon by underlining, coloring, 
highlighting, or by any other means. For example, the method may comprise providing 
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the sender an option to assign a color to the at least typed one word such that the 
chosen emoticon begins to be presented by the animated entity to the recipient at the 
first typed word with the assigned color and the chosen emoticon presentation by the 
animated entity ends at the last typed word with the assigned color" (Column 1 1 , lines 
43-50). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 4, Hyon and Baker do not explicitly teach a display device 
comprising: 

A) wherein when said registered is a gray image, said display control portion converts 
each pixel of said registered image into a color made by mixing the fore color and a 
back color of said text at a ratio according to a pixel value of said predetermined kind of 
character. 

Ostermann, however, teaches "wherein when said registered is a gray 
image, said display control portion converts each pixel of said registered image 
into a color made by mixing the fore color and a back color of said text at a ratio 
according to a pixel value of said predetermined kind of character" as The 

increased intensity of the emoticon may be accomplished by changing the icon from 
black-on-white background to black-on-colored background (such as red or yellow) 
where the intensity of the background color reflects the amplitude" (Column 9, lines 43- 
67) and "The sender can associate typed words with an emoticon by underlining, 
coloring, highlighting, or by any other means. For example, the method may comprise 
providing the sender an option to assign a color to the at least typed one word such that 
the chosen emoticon begins to be presented by the animated entity to the recipient at 
the first typed word with the assigned color and the chosen emoticon presentation by 
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the animated entity ends at the last typed word with the assigned color" (Column 1 1 , 
lines 43-50) 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 5, Hyon and Baker do not explicitly teach a display device 
comprising: 

A) wherein said text attribute data contains decoration attribute data indicating a type of 
a decoration applied to a corresponding predetermined kind of character; and 

B) said display control portion decorates said registered image according to said 
decoration attribute data. 

Ostermann, however, teaches "wherein said text attribute data contains 
decoration attribute data indicating a type of a decoration applied to a 
corresponding predetermined kind of character" as "The sender can associate 
typed words with an emoticon by underlining, coloring, highlighting, or by any other 
means... In this case, in a sentence such as "Hi John, :-) are you pleased that the stock 
market is up?" the underlining represents the highlighting wherein the sender chooses 
to begin the smile at the beginning of the word "are" and to continue the smile through 
the word "up". The method comprises receiving the indicated duration of the emoticon 
and presenting the chosen duration of the emoticon" (Column 11, lines 43-61) and 
"said display control portion decorates said registered image according to said 
decoration attribute data" as "The sender can associate typed words with an 
emoticon by underlining, coloring, highlighting, or by any other means... In this case, in a 
sentence such as "Hi John, :-) are you pleased that the stock market is up?" the 
underlining represents the highlighting wherein the sender chooses to begin the smile at 
the beginning of the word "are" and to continue the smile through the word "up". The 



Application/Control Number: 10/516,881 
Art Unit: 2168 



Page 9 



method comprises receiving the indicated duration of the emoticon and presenting the 
chosen duration of the emoticon" (Column 1 1 , lines 43-61 ). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 6, Hyon teaches a method comprising: 

A) storing an image code for specifying said registered image and registered image 
data corresponding to said image code in a correlated manner according to registration 
processing bv a user (Paragraphs 23, 25, and 44, Figure 2); 

B) said registered image being stored in a storage portion different from a storage 
portion of storing the predetermined kind of character (Paragraphs 23 and 44); 

D) displaying said predetermined kind of character and said registered image 
simultaneously based on said transformed registered image (Paragraphs 21 and 41- 
42). 

The examiner notes that Hyon teaches "storing an image code for specifying 
said registered image and registered image data corresponding to said image 
code in a correlated manner according to registration processing bv a user " as "A 

storage 18 is comprised of a ROM (Read Only Memory) and a RAM (Random Access 
Memory) for storing programs and data, and a voice memory. The storage 18 stores an 
operation program for inputting emoticons and a plurality of emoticons in the form of a 
bit map according to the embodiment of the present invention. The plurality of 
emoticons are stored by groups in the storage 18 to facilitate selection of emoticons" 
(Paragraph 23), "storage 18 stores oriental or occidental emoticons according to the 
cultural area of a user" (Paragraph 25), and "A plurality of emoticons are pre-stored in a 
mobile terminal so that a user easily select an intended emoticon in the present 
invention. In another embodiment, the user can add hieroglyphics to basic emoticons 
stored by a manufacturer. The hieroglyphics can be stored in three ways: the user 
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directly stores them, receives them from a base station and the stores them, or 
downloads them from the Internet. The user can change and edit the emoticons stored 
in the mobile terminal" (Paragraph 44). The examiner further notes that items which are 
clearly not edited/changed/manipulated (i.e. text characters) would be stored in ROM of 
storage 18. Moreover, the examiner further wishes to state that it is well known that text 
characters from a particular font are stored in ROM and not RAM (see cited reference of 
Hersum (see column 4, lines 55-67 for reference). The examiner further notes that 
Hyon teaches "said registered image being stored in a storage portion different 
from a storage portion of storing the predetermined kind of character" as "A 
storage 18 is comprised of a ROM (Read Only Memory) and a RAM (Random Access 
Memory) for storing programs and data, and a voice memory. The storage 18 stores an 
operation program for inputting emoticons and a plurality of emoticons in the form of a 
bit map according to the embodiment of the present invention. The plurality of 
emoticons are stored by groups in the storage 18 to facilitate selection of emoticons" 
(Paragraph 23) and "In another embodiment, the user can add hieroglyphics to basic 
emoticons stored by a manufacturer. The hieroglyphics can be stored in three ways: 
the user directly stores them, receives them from a base station, and the stores them, or 
downloads them from the Internet" (Paragraph 44). The examiner further notes that it is 
clear that the emoticons which are edited, changed, and user-generated must be stored 
in the RAM of storage 18 since items stored in RAM can be written to whereas items 
stored in ROM cannot be written (i.e. are Read-Only). The examiner further notes that 
Hyon teaches "displaying said predetermined kind of character and said 
registered image simultaneously based on said transformed registered image" as 
"a display 16 outputs display data and text messages generated in the mobile terminal. 
An LCD (Liquid Crystal Display) can be used as the display 16" (Paragraph 21), "If the 
user selects "smile" in the display 6 by manipulating a left/right directional key, 
emoticons that belong to the group "smile" are displayed in a display 7. When the user 
chooses one of the "smile" emoticons " . " in the display 7, the selected emoticon is 
stored as part of the short message as shown in a display 8. Then, the user can input a 
text including typical characters, special characters, or emoticons within the range of a 
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transmittable SMS message, by changing input modes" (Paragraph 41), and "After an 
intended text is completed, the user instructs the mobile terminal to transmit the SMS 
message including the emoticon...the mobile terminal transmits the stored message" 
(Paragraph 42). 

Hyon does not explicitly teach: 
C) with respect to display data containing a series of a character code, text attribute 
data, and said image code, transforming said registered image to be displayed 
according to said text attribute data; 

F) by determining whether or not a size of characters constituting the text where the 
registered image is inserted and a size of the registered image are substantially the 
same : 

G) when the size of characters constituting the text where the registered image is 
inserted and the size of the registered image are not substantially the same, said 
display control portion makes the size of the characters and the size of the registered 
image substantially the same bv scaling up/down the size of the registered image based 
on the text attribute data : 

I) causing said display output portion to display a text containing characters scaled to 
substantially the same size as the registered image . 

Baker, however, teaches "with respect to display data containing a series of 
a character code, text attribute data, and said image code, transforming said 
registered image to be displayed according to said text attribute data" as "in order 
to accommodate the use of different size fonts in the mailbox display, means for scaling 
the size of the icon graphics are also provided... at least one image for each icon is 
stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40), " bv 
determining whether or not a size of characters constituting the text where the 
registered image is inserted and a size of the registered image are substantially 
the same " as "in order to accommodate the use of different size fonts in the mailbox 
display, means for scaling the size of the icon graphics are also provided... at least one 
image for each icon is stored, the icon most closely matching the point size of the font is 
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chosen and then scaled as needed to better match the font point size" (Column 8, lines 
26-40), " when the size of characters constituting the text where the registered 
image is inserted and the size of the registered image are not substantially the 
same, said display control portion makes the size of the characters and the size 
of the registered image substantially the same by scaling up/down the size of the 
registered image based on the text attribute data " as "in order to accommodate the 
use of different size fonts in the mailbox display, means for scaling the size of the icon 
graphics are also provided... at least one image for each icon is stored, the icon most 
closely matching the point size of the font is chosen and then scaled as needed to 
better match the font point size" (Column 8, lines 26-40), and " causing said display 
output portion to display a text containing characters scaled to substantially the 
same size as the registered image " as "in order to accommodate the use of different 
size fonts in the mailbox display, means for scaling the size of the icon graphics are also 
provided... at least one image for each icon is stored, the icon most closely matching the 
point size of the font is chosen and then scaled as needed to better match the font point 
size" (Column 8, lines 26-40). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Baker's would have allowed Hyon's to provide a method for allowing for scalable icons 
to accompany texts with specified and varied fonts in order to easily associate an icon 
to a particular sentence, as noted by Baker (Column 5, lines 40-55). 

Hyon and Baker do not explicitly teach: 
C) said text attribute data containing color attribute data indicating at least a fore color 
of a corresponding predetermined kind of character : 

E) so that said registered image is sandwiched between two parts of said text of said 

predetermined kind of character in a line of said identical sentence; 

H) converting a color of said registered image according to said color attribute data . 

Ostermann, however, teaches " said text attribute data containing color 
attribute data indicating at least a fore color of a corresponding predetermined 
kind of character " as "The increased intensity of the emoticon may be accomplished 
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by changing the icon from black-on-white background to black-on-colored background 
(such as red or yellow) where the intensity of the background color reflects the 
amplitude" (Column 9, lines 43-67) and "The sender can associate typed words with an 
emoticon by underlining, coloring, highlighting, or by any other means. For example, 
the method may comprise providing the sender an option to assign a color to the at 
least typed one word such that the chosen emoticon begins to be presented by the 
animated entity to the recipient at the first typed word with the assigned color and the 
chosen emoticon presentation by the animated entity ends at the last typed word with 
the assigned color" (Column 1 1 , lines 43-50), "so that said registered image is 
sandwiched between two parts of said text of said predetermined kind of 
character in a line of said identical sentence" as "The sender may also insert 
emoticons 103 into the text of the message. The system includes predefined emoticons 
96, such as ":-) " for a smile, "::-)" for a head nod, "*w*" for an eye wink, and so forth. 
The predefined emoticons are represented either as icons or as text, such as ";-)"■ As 
shown in FIG. 6, the window 80 includes a sample group of emoticon icons 96. The 
sender inserts an emoticon into a text message at the location of the cursor 102 by 
clicking on one of the emoticon icons 100. The sender may also type in the desired 
emoticon as text. Emoticon icons 96 save the sender from needing to type three keys, 
such as ":" and "-" and ")" for a smile. The icons 96 may be either a picture of, say, a 
winking eye or a icon representation of the characters ";-)" 1 00, or other information 
indicating to the sender that clicking on that emoticon icon will insert the associated 
emotion 103 into the text at the location of the cursor 102" (Column 8, lines 35-50), and 
" converting a color of said registered image according to said color attribute 
data " as "The increased intensity of the emoticon may be accomplished by changing 
the icon from black-on-white background to black-on-colored background (such as red 
or yellow) where the intensity of the background color reflects the amplitude" (Column 9, 
lines 43-67) and "The sender can associate typed words with an emoticon by 
underlining, coloring, highlighting, or by any other means. For example, the method 
may comprise providing the sender an option to assign a color to the at least typed one 
word such that the chosen emoticon begins to be presented by the animated entity to 
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the recipient at the first typed word with the assigned color and the chosen emoticon 
presentation by the animated entity ends at the last typed word with the assigned color" 
(Column 11, lines 43-50). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 7, Hyon teaches a viewer program comprising: 

A) storing an image code for specifying said registered image and registered image 
data corresponding to said image code in a correlated manner according to registration 
processing by a user (Paragraphs 23, 25, and 44, Figure 2); 

B) said registered image being stored in a storage portion different from a storage 
portion of storing the predetermined kind of character (Paragraphs 23 and 44); 

D) displaying said predetermined kind of character and said registered image 
simultaneously based on said transformed registered image (Paragraphs 21 and 41- 
42). 

The examiner notes that Hyon teaches "storing an image code for specifying 
said registered image and registered image data corresponding to said image 
code in a correlated manner according to registration processing bv a user " as "A 

storage 18 is comprised of a ROM (Read Only Memory) and a RAM (Random Access 
Memory) for storing programs and data, and a voice memory. The storage 18 stores an 
operation program for inputting emoticons and a plurality of emoticons in the form of a 
bit map according to the embodiment of the present invention. The plurality of 
emoticons are stored by groups in the storage 18 to facilitate selection of emoticons" 
(Paragraph 23), "storage 18 stores oriental or occidental emoticons according to the 
cultural area of a user" (Paragraph 25), and "A plurality of emoticons are pre-stored in a 
mobile terminal so that a user easily select an intended emoticon in the present 
invention. In another embodiment, the user can add hieroglyphics to basic emoticons 
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stored by a manufacturer. The hieroglyphics can be stored in three ways: the user 
directly stores them, receives them from a base station and the stores them, or 
downloads them from the Internet. The user can change and edit the emoticons stored 
in the mobile terminal" (Paragraph 44). The examiner further notes that items which are 
clearly not edited/changed/manipulated (i.e. text characters) would be stored in ROM of 
storage 18. Moreover, the examiner further wishes to state that it is well known that text 
characters from a particular font are stored in ROM and not RAM (see cited reference of 
Hersum (see column 4, lines 55-67 for reference). The examiner further notes that 
Hyon teaches "said registered image being stored in a storage portion different 
from a storage portion of storing the predetermined kind of character" as "A 
storage 18 is comprised of a ROM (Read Only Memory) and a RAM (Random Access 
Memory) for storing programs and data, and a voice memory. The storage 18 stores an 
operation program for inputting emoticons and a plurality of emoticons in the form of a 
bit map according to the embodiment of the present invention. The plurality of 
emoticons are stored by groups in the storage 18 to facilitate selection of emoticons" 
(Paragraph 23) and "In another embodiment, the user can add hieroglyphics to basic 
emoticons stored by a manufacturer. The hieroglyphics can be stored in three ways: 
the user directly stores them, receives them from a base station, and the stores them, or 
downloads them from the Internet" (Paragraph 44). The examiner further notes that it is 
clear that the emoticons which are edited, changed, and user-generated must be stored 
in the RAM of storage 18 since items stored in RAM can be written to whereas items 
stored in ROM cannot be written (i.e. are Read-Only). The examiner further notes that 
Hyon teaches "displaying said predetermined kind of character and said 
registered image simultaneously based on said transformed registered image" as 
"a display 16 outputs display data and text messages generated in the mobile terminal. 
An LCD (Liquid Crystal Display) can be used as the display 16" (Paragraph 21), "If the 
user selects "smile" in the display 6 by manipulating a left/right directional key, 
emoticons that belong to the group "smile" are displayed in a display 7. When the user 
chooses one of the "smile" emoticons " . " in the display 7, the selected emoticon is 
stored as part of the short message as shown in a display 8. Then, the user can input a 
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text including typical characters, special characters, or emoticons within the range of a 
transmittable SMS message, by changing input modes" (Paragraph 41), and "After an 
intended text is completed, the user instructs the mobile terminal to transmit the SMS 
message including the emoticon...the mobile terminal transmits the stored message" 
(Paragraph 42). The examiner further notes that it is common knowledge that when text 
messages are received via a device, the entire message is displayed at once. The 
examiner further notes that Hyon teaches "when displaying a vertical writing" as 
"Emoticons are usually divided into two categories: oriental ones used among Asians 
including Koreans and Japanese, and occidental ones mainly used in the U.S. and 
Europe. Vertical hieroglyphics prevail in the former case, whereas horizontal ones 
prevail in the latter case. Therefore, storage 18 stores oriental or occidental emoticons 
according to the cultural area of a user" (Paragraph 25). 

Hyon does not explicitly teach: 
C) with respect to display data containing a series of a character code, text attribute 
data, and said image code, transforming said registered image to be displayed 
according to said text attribute data; 

F) by determining whether or not a size of characters constituting the text where the 
registered image is inserted and a size of the registered image are substantially the 
same : 

G) when the size of characters constituting the text where the registered image is 
inserted and the size of the registered image are not substantially the same, said 
display control portion makes the size of the characters and the size of the registered 
image substantially the same by scaling up/down the size of the registered image based 
on the text attribute data : and 

I) causing said display output portion to display a text containing characters scaled to 
substantially the same size as the registered image . 

Baker, however, teaches "with respect to display data containing a series of 
a character code, text attribute data, and said image code, transforming said 
registered image to be displayed according to said text attribute data" as "in order 
to accommodate the use of different size fonts in the mailbox display, means for scaling 
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the size of the icon graphics are also provided... at least one image for each icon is 
stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40), " by 
determining whether or not a size of characters constituting the text where the 
registered image is inserted and a size of the registered image are substantially 
the same " as "in order to accommodate the use of different size fonts in the mailbox 
display, means for scaling the size of the icon graphics are also provided... at least one 
image for each icon is stored, the icon most closely matching the point size of the font is 
chosen and then scaled as needed to better match the font point size" (Column 8, lines 
26-40), " when the size of characters constituting the text where the registered 
image is inserted and the size of the registered image are not substantially the 
same, said display control portion makes the size of the characters and the size 
of the registered image substantially the same by scaling up/down the size of the 
registered image based on the text attribute data " as "in order to accommodate the 
use of different size fonts in the mailbox display, means for scaling the size of the icon 
graphics are also provided... at least one image for each icon is stored, the icon most 
closely matching the point size of the font is chosen and then scaled as needed to 
better match the font point size" (Column 8, lines 26-40), and " causing said display 
output portion to display a text containing characters scaled to substantially the 
same size as the registered image " as "in order to accommodate the use of different 
size fonts in the mailbox display, means for scaling the size of the icon graphics are also 
provided... at least one image for each icon is stored, the icon most closely matching the 
point size of the font is chosen and then scaled as needed to better match the font point 
size" (Column 8, lines 26-40). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Baker's would have allowed Hyon's to provide a method for allowing for scalable icons 
to accompany texts with specified and varied fonts in order to easily associate an icon 
to a particular sentence, as noted by Baker (Column 5, lines 40-55). 

Hyon and Baker do not explicitly teach: 
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C) said text attribute data containing color attribute data indicating at least a fore color 
of a corresponding predetermined kind of character : 

E) so that said registered image is sandwiched between two parts of said text of said 

predetermined kind of character in a line of said identical sentence; 

H) converting a color of said registered image according to said color attribute data . 

Ostermann, however, teaches " said text attribute data containing color 
attribute data indicating at least a fore color of a corresponding predetermined 
kind of character " as "The increased intensity of the emoticon may be accomplished 
by changing the icon from black-on-white background to black-on-colored background 
(such as red or yellow) where the intensity of the background color reflects the 
amplitude" (Column 9, lines 43-67) and "The sender can associate typed words with an 
emoticon by underlining, coloring, highlighting, or by any other means. For example, 
the method may comprise providing the sender an option to assign a color to the at 
least typed one word such that the chosen emoticon begins to be presented by the 
animated entity to the recipient at the first typed word with the assigned color and the 
chosen emoticon presentation by the animated entity ends at the last typed word with 
the assigned color" (Column 1 1 , lines 43-50), "so that said registered image is 
sandwiched between two parts of said text of said predetermined kind of 
character in a line of said identical sentence" as "The sender may also insert 
emoticons 103 into the text of the message. The system includes predefined emoticons 
96, such as ":-) " for a smile, "::-)" for a head nod, "*w*" for an eye wink, and so forth. 
The predefined emoticons are represented either as icons or as text, such as ";-)". As 
shown in FIG. 6, the window 80 includes a sample group of emoticon icons 96. The 
sender inserts an emoticon into a text message at the location of the cursor 102 by 
clicking on one of the emoticon icons 100. The sender may also type in the desired 
emoticon as text. Emoticon icons 96 save the sender from needing to type three keys, 
such as ":" and "-" and ")" for a smile. The icons 96 may be either a picture of, say, a 
winking eye or a icon representation of the characters ";-)" 100, or other information 
indicating to the sender that clicking on that emoticon icon will insert the associated 
emotion 103 into the text at the location of the cursor 102" (Column 8, lines 35-50), and 
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" converting a color of said registered image according to said color attribute 
data, and causing said display output portion to display a text containing 
characters scaled to substantially the same size as the registered image " as "The 
increased intensity of the emoticon may be accomplished by changing the icon from 
black-on-white background to black-on-colored background (such as red or yellow) 
where the intensity of the background color reflects the amplitude" (Column 9, lines 43- 
67) and "The sender can associate typed words with an emoticon by underlining, 
coloring, highlighting, or by any other means. For example, the method may comprise 
providing the sender an option to assign a color to the at least typed one word such that 
the chosen emoticon begins to be presented by the animated entity to the recipient at 
the first typed word with the assigned color and the chosen emoticon presentation by 
the animated entity ends at the last typed word with the assigned color" (Column 1 1 , 
lines 43-50). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 8, Hyon teaches a computer readable recording medium 
comprising: 

A) storing an image code for specifying said registered image and registered image 
data corresponding to said image code in a correlated manner according to registration 
processing by a user (Paragraphs 23, 25, and 44, Figure 2); 

B) said registered image stored in a storage portion different from a storage portion of 
storing the predetermined kind of character (Paragraphs 23 and 44); 

D) displaying said predetermined kind of character and said registered image 
simultaneously based on said transformed registered image (Paragraphs 21 and 41- 
42). 
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The examiner notes that Hyon teaches "storing an image code for specifying 
said registered image and registered image data corresponding to said image 
code in a correlated manner according to registration processing by a user " as "A 

storage 18 is comprised of a ROM (Read Only Memory) and a RAM (Random Access 
Memory) for storing programs and data, and a voice memory. The storage 18 stores an 
operation program for inputting emoticons and a plurality of emoticons in the form of a 
bit map according to the embodiment of the present invention. The plurality of 
emoticons are stored by groups in the storage 18 to facilitate selection of emoticons" 
(Paragraph 23), "storage 18 stores oriental or occidental emoticons according to the 
cultural area of a user" (Paragraph 25), and "A plurality of emoticons are pre-stored in a 
mobile terminal so that a user easily select an intended emoticon in the present 
invention. In another embodiment, the user can add hieroglyphics to basic emoticons 
stored by a manufacturer. The hieroglyphics can be stored in three ways: the user 
directly stores them, receives them from a base station and the stores them, or 
downloads them from the Internet. The user can change and edit the emoticons stored 
in the mobile terminal" (Paragraph 44). The examiner further notes that it is clear that 
the emoticons which are edited, changed, and user-generated must be stored in the 
RAM of storage 18 since items stored in RAM can be written to whereas items stored in 
ROM cannot be written (i.e. are Read-Only). The examiner further notes that items 
which are clearly not edited/changed/manipulated (i.e. text characters) would be stored 
in ROM of storage 1 8. Moreover, the examiner further wishes to state that it is well 
known that text characters from a particular font are stored in ROM and not RAM (see 
cited reference of Hersum (see column 4, lines 55-67 for reference). The examiner 
further notes that Hyon teaches "said registered image stored in a storage portion 
different from a storage portion of storing the predetermined kind of character" 
as "A storage 18 is comprised of a ROM (Read Only Memory) and a RAM (Random 
Access Memory) for storing programs and data, and a voice memory. The storage 18 
stores an operation program for inputting emoticons and a plurality of emoticons in the 
form of a bit map according to the embodiment of the present invention. The plurality of 
emoticons are stored by groups in the storage 18 to facilitate selection of emoticons" 
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(Paragraph 23) and "In another embodiment, the user can add hieroglyphics to basic 
emoticons stored by a manufacturer. The hieroglyphics can be stored in three ways: 
the user directly stores them, receives them from a base station, and the stores them, or 
downloads them from the Internet" (Paragraph 44). The examiner further notes that 
Hyon teaches "displaying said predetermined kind of character and said 
registered image simultaneously based on said transformed registered image" as 
"a display 16 outputs display data and text messages generated in the mobile terminal. 
An LCD (Liquid Crystal Display) can be used as the display 16" (Paragraph 21), "If the 
user selects "smile" in the display 6 by manipulating a left/right directional key, 
emoticons that belong to the group "smile" are displayed in a display 7. When the user 
chooses one of the "smile" emoticons " . " in the display 7, the selected emoticon is 
stored as part of the short message as shown in a display 8. Then, the user can input a 
text including typical characters, special characters, or emoticons within the range of a 
transmittable SMS message, by changing input modes" (Paragraph 41), and "After an 
intended text is completed, the user instructs the mobile terminal to transmit the SMS 
message including the emoticon... the mobile terminal transmits the stored message" 
(Paragraph 42). The examiner further notes that it is common knowledge that when text 
messages are received via a device, the entire message is displayed at once. The 
examiner further notes that Hyon teaches "when displaying a vertical writing" as 
"Emoticons are usually divided into two categories: oriental ones used among Asians 
including Koreans and Japanese, and occidental ones mainly used in the U.S. and 
Europe. Vertical hieroglyphics prevail in the former case, whereas horizontal ones 
prevail in the latter case. Therefore, storage 18 stores oriental or occidental emoticons 
according to the cultural area of a user" (Paragraph 25). 

Hyon does not explicitly teach: 
C) with respect to display data containing a series of a character code, text attribute 
data, and said image code, transforming said registered image to be displayed 
according to said text attribute data; 
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F) by determining whether or not a size of characters constituting the text where the 
registered image is inserted and a size of the registered image are substantially the 
same : 

G) when the size of characters constituting the text where the registered image is 
inserted and the size of the registered image are not substantially the same, said 
display control portion makes the size of the characters and the size of the registered 
image substantially the same by scaling up/down the size of the registered image based 
on the text attribute data : and 

I) causing said display output portion to display a text containing characters scaled to 
substantially the same size as the registered image . 

Baker, however, teaches "with respect to display data containing a series of 
a character code, text attribute data, and said image code, transforming said 
registered image to be displayed according to said text attribute data" as "in order 
to accommodate the use of different size fonts in the mailbox display, means for scaling 
the size of the icon graphics are also provided... at least one image for each icon is 
stored, the icon most closely matching the point size of the font is chosen and then 
scaled as needed to better match the font point size" (Column 8, lines 26-40), " by 
determining whether or not a size of characters constituting the text where the 
registered image is inserted and a size of the registered image are substantially 
the same " as "in order to accommodate the use of different size fonts in the mailbox 
display, means for scaling the size of the icon graphics are also provided... at least one 
image for each icon is stored, the icon most closely matching the point size of the font is 
chosen and then scaled as needed to better match the font point size" (Column 8, lines 
26-40), " when the size of characters constituting the text where the registered 
image is inserted and the size of the registered image are not substantially the 
same, said display control portion makes the size of the characters and the size 
of the registered image substantially the same by scaling up/down the size of the 
registered image based on the text attribute data " as "in order to accommodate the 
use of different size fonts in the mailbox display, means for scaling the size of the icon 
graphics are also provided... at least one image for each icon is stored, the icon most 
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closely matching the point size of the font is chosen and then scaled as needed to 
better match the font point size" (Column 8, lines 26-40), and " causing said display 
output portion to display a text containing characters scaled to substantially the 
same size as the registered image " as "in order to accommodate the use of different 
size fonts in the mailbox display, means for scaling the size of the icon graphics are also 
provided... at least one image for each icon is stored, the icon most closely matching the 
point size of the font is chosen and then scaled as needed to better match the font point 
size" (Column 8, lines 26-40). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Baker's would have allowed Hyon's to provide a method for allowing for scalable icons 
to accompany texts with specified and varied fonts in order to easily associate an icon 
to a particular sentence, as noted by Baker (Column 5, lines 40-55). 

Hyon and Baker do not explicitly teach: 
C) said text attribute data containing color attribute data indicating at least a fore color 
of a corresponding predetermined kind of character : 

E) so that said registered image is sandwiched between two parts of said text of said 

predetermined kind of character in a line of said identical sentence; 

H) converting a color of said registered image according to said color attribute data . 

Ostermann, however, teaches " said text attribute data containing color 
attribute data indicating at least a fore color of a corresponding predetermined 
kind of character " as "The increased intensity of the emoticon may be accomplished 
by changing the icon from black-on-white background to black-on-colored background 
(such as red or yellow) where the intensity of the background color reflects the 
amplitude" (Column 9, lines 43-67) and "The sender can associate typed words with an 
emoticon by underlining, coloring, highlighting, or by any other means. For example, 
the method may comprise providing the sender an option to assign a color to the at 
least typed one word such that the chosen emoticon begins to be presented by the 
animated entity to the recipient at the first typed word with the assigned color and the 
chosen emoticon presentation by the animated entity ends at the last typed word with 
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the assigned color" (Column 1 1 , lines 43-50), "so that said registered image is 
sandwiched between two parts of said text of said predetermined kind of 
character in a line of said identical sentence" as "The sender may also insert 
emoticons 103 into the text of the message. The system includes predefined emoticons 
96, such as ":-) " for a smile, "::-)" for a head nod, "*w*" for an eye wink, and so forth. 
The predefined emoticons are represented either as icons or as text, such as ";-)". As 
shown in FIG. 6, the window 80 includes a sample group of emoticon icons 96. The 
sender inserts an emoticon into a text message at the location of the cursor 102 by 
clicking on one of the emoticon icons 100. The sender may also type in the desired 
emoticon as text. Emoticon icons 96 save the sender from needing to type three keys, 
such as ":" and "-" and ")" for a smile. The icons 96 may be either a picture of, say, a 
winking eye or a icon representation of the characters ";-)" 100, or other information 
indicating to the sender that clicking on that emoticon icon will insert the associated 
emotion 103 into the text at the location of the cursor 102" (Column 8, lines 35-50), and 
"converting a color of said registered image according to said color attribute 
data, and causing said display output portion to display a text containing 
characters scaled to substantially the same size as the registered image " as "The 
increased intensity of the emoticon may be accomplished by changing the icon from 
black-on-white background to black-on-colored background (such as red or yellow) 
where the intensity of the background color reflects the amplitude" (Column 9, lines 43- 
67) and "The sender can associate typed words with an emoticon by underlining, 
coloring, highlighting, or by any other means. For example, the method may comprise 
providing the sender an option to assign a color to the at least typed one word such that 
the chosen emoticon begins to be presented by the animated entity to the recipient at 
the first typed word with the assigned color and the chosen emoticon presentation by 
the animated entity ends at the last typed word with the assigned color" (Column 1 1 , 
lines 43-50). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
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the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 9, Hyon and Baker do not explicitly teach a display device 
comprising: 

A) wherein said display control portion includes image decoration means for decorating 
said registered image to provide continuous decoration to both said registered image 
and neighboring predetermined kind of characters adjacent to said registered image 
according to said decoration attribute data. 

Ostermann, however, teaches "wherein said display control portion includes 
image decoration means for decorating said registered image to provide 
continuous decoration to both said registered image and neighboring 
predetermined kind of characters adjacent to said registered image according to 
said decoration attribute data" as "The sender can associate typed words with an 
emoticon by underlining, coloring, highlighting, or by any other means... In this case, in a 
sentence such as "Hi John, :-) are you pleased that the stock market is up?" the 
underlining represents the highlighting wherein the sender chooses to begin the smile at 
the beginning of the word "are" and to continue the smile through the word "up". The 
method comprises receiving the indicated duration of the emoticon and presenting the 
chosen duration of the emoticon as the animated entity delivers the message. As 
mentioned above, the highlighting can occur through coloring words, underlining words, 
or some other means of presenting the emotion" (Column 1 1 , lines 43-61 ). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 10, Hyon and Baker do not explicitly teach a display device 
comprising: 
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A) wherein said image decoration means decorates said registered image, such that 
both said registered image and said neighboring predetermined kind of characters show 
the said fore and back colors continuously. 

Ostermann, however, teaches "wherein said image decoration means 
decorates said registered image, such that both said registered image and said 
neighboring predetermined kind of characters show the said fore and back colors 
continuously" as The increased intensity of the emoticon may be accomplished by 
changing the icon from black-on-white background to black-on-colored background 
(such as red or yellow) where the intensity of the background color reflects the 
amplitude" (Column 9, lines 43-67) and "The sender can associate typed words with an 
emoticon by underlining, coloring, highlighting, or by any other means... In this case, in a 
sentence such as "Hi John, :-) are you pleased that the stock market is up?" the 
underlining represents the highlighting wherein the sender chooses to begin the smile at 
the beginning of the word "are" and to continue the smile through the word "up". The 
method comprises receiving the indicated duration of the emoticon and presenting the 
chosen duration of the emoticon as the animated entity delivers the message. As 
mentioned above, the highlighting can occur through coloring words, underlining words, 
or some other means of presenting the emotion" (Column 1 1 , lines 43-61 ). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 1 1 , Hyon and Baker do not explicitly teach a display device 
comprising: 

A) wherein said image decoration means decorates said registered image to draw a 
continuous underline below both said registered image and said neighboring 
predetermined kind of characters. 
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Ostermann, however, teaches "wherein said image decoration means 
decorates said registered image to draw a continuous underline below both said 
registered image and said neighboring predetermined kind of characters" as "The 
sender can associate typed words with an emoticon by underlining, coloring, 
highlighting, or by any other means... In this case, in a sentence such as "Hi John, :-) 
are you pleased that the stock market is up?" the underlining represents the highlighting 
wherein the sender chooses to begin the smile at the beginning of the word "are" and to 
continue the smile through the word "up". The method comprises receiving the 
indicated duration of the emoticon and presenting the chosen duration of the emoticon 
as the animated entity delivers the message. As mentioned above, the highlighting can 
occur through coloring words, underlining words, or some other means of presenting the 
emotion" (Column 1 1 , lines 43-61 ). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 12, Hyon and Baker do not explicitly teach a display device 
comprising: 

A) wherein said image decoration means decorates said registered image to draw a 
continuous cancel line on both said registered image and said neighboring 
predetermined kind of characters. 

Ostermann, however, teaches "wherein said image decoration means 
decorates said registered image to draw a continuous cancel line on both said 
registered image and said neighboring predetermined kind of characters" as "The 
sender can associate typed words with an emoticon by underlining, coloring, 
highlighting, or by any other means.. .In this case, in a sentence such as "Hi John, :-) 
are you pleased that the stock market is up?" the underlining represents the highlighting 
wherein the sender chooses to begin the smile at the beginning of the word "are" and to 
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continue the smile through the word "up". The method comprises receiving the 
indicated duration of the emoticon and presenting the chosen duration of the emoticon 
as the animated entity delivers the message. As mentioned above, the highlighting can 
occur through coloring words, underlining words, or some other means of presenting the 
emotion" (Column 1 1 , lines 43-61 ). 

The examiner notes that it is clear that Ostermann's method can use various 
computer font techniques to apply to sentences with emoticons, such as underlines and 
highlighters. 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 13, Hyon further teaches a display device comprising: 

A) wherein the first storage portion is a ROM (Paragraph 23); and 

B) the second storage portion is a RAM (Paragraphs 23 and 44). 

The examiner notes that Hyon teaches "wherein the first storage portion is a 
ROM" as "A storage 18 is comprised of a ROM (Read Only Memory) and a RAM 
(Random Access Memory) for storing programs and data, and a voice memory. The 
storage 18 stores an operation program for inputting emoticons and a plurality of 
emoticons in the form of a bit map according to the embodiment of the present 
invention. The plurality of emoticons are stored by groups in the storage 18 to facilitate 
selection of emoticons" (Paragraph 23). The examiner further notes that items which 
are clearly not edited/changed/manipulated (i.e. text characters) would be stored in 
ROM of storage 18. Moreover, the examiner further wishes to state that it is well known 
that text characters from a particular font are stored in ROM and not RAM (see cited 
reference of Hersum (see column 4, lines 55-67 for reference). The examiner further 
notes that Hyon teaches "the second storage portion is a RAM" as "A storage 18 is 
comprised of a ROM (Read Only Memory) and a RAM (Random Access Memory) for 
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storing programs and data, and a voice memory. The storage 18 stores an operation 
program for inputting emoticons and a plurality of emoticons in the form of a bit map 
according to the embodiment of the present invention. The plurality of emoticons are 
stored by groups in the storage 18 to facilitate selection of emoticons" (Paragraph 23) 
and "A plurality of emoticons are pre-stored in a mobile terminal so that a user easily 
select an intended emoticon in the present invention. In another embodiment, the user 
can add hieroglyphics to basic emoticons stored by a manufacturer. The hieroglyphics 
can be stored in three ways: the user directly stores them, receives them from a base 
station and the stores them, or downloads them from the Internet. The user can change 
and edit the emoticons stored in the mobile terminal" (Paragraph 44). The examiner 
further notes that it is clear that the emoticons which are edited, changed, and user- 
generated must be stored in the RAM of storage 18 since items stored in RAM can be 
written to whereas items stored in ROM cannot be written (i.e. are Read-Only). 

Regarding claims 14-17, Hyon and Baker do not explicitly teach a display 
device, display method, viewer program, and computer-readable recording medium 
comprising: 

A) wherein the text containing the predetermined kind of character and the registered 
image is used in a Hyper Text Markup Language document. 

Ostermann, however, teaches "wherein the text containing the 
predetermined kind of character and the registered image is used in a Hyper Text 
Markup Language document" as "FIG. 4(a) illustrates a high-bandwidth architecture 
60 associated with the embodiments of the invention. The system 60 delivers a hyper- 
text mark-up language (HTML) page through the Internet 62 (connected to a web 
server, not shown but embodied in the Internet 62) to a client application 64. The HTML 
page (shown by way of example in FIG. 6) enables the sender to create a multi-media 
message. The client application may be, for example, a web browser such as 
Microsoft's Internet Explorer. RTM.. Other client applications include e-mail and instant 
messaging clients. The sender creates the multi-media message using the HTML page" 
(Column 4, lines 53-63). 
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The examiner notes that it is clear that Ostermann's method can use various 
computer font techniques to apply to sentences with emoticons, such as underlines and 
highlighters. 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Regarding claim 19, Hyon teaches a method comprising: 

A) storing beforehand a character code for specifying each of said predetermined kind 
of character , and character shape data corresponding to said character in a correlated 
manner (Paragraphs 23, and 41-42, Figure 4); 

B) storing an image code for specifying said registered image and registered image 
data corresponding to said image code in a correlated manner according to registration 
processing bv a user (Paragraphs 23, 25, and 44, Figure 2); 

C) outputting said predetermined kind of character and said registered image 
(Paragraph 21 , Figure 2); and 

D) causing said display output portion to output corresponding said predetermined kind 
of character and said registered image both belonging to an identical sentence based 
on display data containing a series of said character code, text attribute data, and said 
image code (Paragraphs 21 and 41-42). 

The examiner notes that Hyon teaches "storing beforehand a character code 
for specifying each of said predetermined kind of characte r, and character shape 
data corresponding to said character in a correlated manner " as "A storage 1 8 is 
comprised of a ROM (Read Only Memory) and a RAM (Random Access Memory) for 
storing programs and data, and a voice memory. The storage 18 stores an operation 
program for inputting emoticons and a plurality of emoticons in the form of a bit map 
according to the embodiment of the present invention. The plurality of emoticons are 
stored by groups in the storage 18 to facilitate selection of emoticons" (Paragraph 23) 
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and "the user can input a text including typical characters, special characters, or 
emoticons within the range of a transmittable SMS message, by changing input modes" 
(Paragraph 41 ). The examiner further notes that Hyon teaches "storing an image 
code for specifying said registered image and registered image data 
corresponding to said image code in a correlated manner according to 
registration processing by a user " as "A storage 1 8 is comprised of a ROM (Read 
Only Memory) and a RAM (Random Access Memory) for storing programs and data, 
and a voice memory. The storage 18 stores an operation program for inputting 
emoticons and a plurality of emoticons in the form of a bit map according to the 
embodiment of the present invention. The plurality of emoticons are stored by groups in 
the storage 18 to facilitate selection of emoticons" (Paragraph 23), "storage 18 stores 
oriental or occidental emoticons according to the cultural area of a user" (Paragraph 
25), and "A plurality of emoticons are pre-stored in a mobile terminal so that a user 
easily select an intended emoticon in the present invention. In another embodiment, the 
user can add hieroglyphics to basic emoticons stored by a manufacturer. The 
hieroglyphics can be stored in three ways: the user directly stores them, receives them 
from a base station and the stores them, or downloads them from the Internet. The user 
can change and edit the emoticons stored in the mobile terminal" (Paragraph 44). The 
examiner further notes that it is clear that the emoticons which are edited, changed, and 
user-generated must be stored in the RAM of storage 1 8 since items stored in RAM can 
be written to whereas items stored in ROM cannot be written (i.e. are Read-Only). The 
examiner further notes that Hyon teaches "outputting said predetermined kind of 
character and said registered image" as "a display 16 outputs display data and text 
messages generated in the mobile terminal. An LCD (Liquid Crystal Display) can be 
used as the display 1 6" (Paragraph 21 ). The examiner further notes that Hyon teaches 
"causing said display output portion to output corresponding said predetermined 
kind of character and said registered image both belonging to an identical 
sentence based on display data containing a series of said character code, text 
attribute data, and said image code" as "a display 16 outputs display data and text 
messages generated in the mobile terminal. An LCD (Liquid Crystal Display) can be 
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used as the display 16" (Paragraph 21), "If the user selects "smile" in the display 6 by 
manipulating a left/right directional key, emoticons that belong to the group "smile" are 
displayed in a display 7. When the user chooses one of the "smile" emoticons " . " in the 
display 7, the selected emoticon is stored as part of the short message as shown in a 
display 8. Then, the user can input a text including typical characters, special 
characters, or emoticons within the range of a transmittable SMS message, by changing 
input modes" (Paragraph 41 ), and "After an intended text is completed, the user 
instructs the mobile terminal to transmit the SMS message including the emoticon... the 
mobile terminal transmits the stored message" (Paragraph 42). The examiner further 
notes that it is common knowledge that when text messages are received via a device, 
the entire message is displayed at once. 
Hyon does not explicitly teach: 

F) determining whether or not size of characters constituting the text where the 
registered image is inserted and a size of the registered image are substantially the 
same : 

G) when the size of characters constituting the text where the registered image is 
inserted and the size of the registered image are not substantially the same, said 
display control portion makes the size of the characters and the size of the registered 
image substantially the same by scaling up/down the size of the registered image based 
on the text attribute data : and 

I) said display control portion causes said display output portion to display a text 
containing characters and the registered image scaled to substantially the same size . 

Baker, however, teaches " determining whether or not size of characters 
constituting the text where the registered image is inserted and a size of the 
registered image are substantially the same " as "in order to accommodate the use of 
different size fonts in the mailbox display, means for scaling the size of the icon 
graphics are also provided... at least one image for each icon is stored, the icon most 
closely matching the point size of the font is chosen and then scaled as needed to 
better match the font point size" (Column 8, lines 26-40), " when the size of characters 
constituting the text where the registered image is inserted and the size of the 
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registered image are not substantially the same, said display control portion 
makes the size of the characters and the size of the registered image 
substantially the same bv scaling up/down the size of the registered image based 
on the text attribute data " as "in order to accommodate the use of different size fonts 
in the mailbox display, means for scaling the size of the icon graphics are also 
provided... at least one image for each icon is stored, the icon most closely matching the 
point size of the font is chosen and then scaled as needed to better match the font point 
size" (Column 8, lines 26-40), and " said display control portion causes said display 
output portion to display a text containing characters and the registered image 
scaled to substantially the same size " as "in order to accommodate the use of 
different size fonts in the mailbox display, means for scaling the size of the icon 
graphics are also provided... at least one image for each icon is stored, the icon most 
closely matching the point size of the font is chosen and then scaled as needed to 
better match the font point size" (Column 8, lines 26-40). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Baker's would have allowed Hyon's to provide a method for allowing for scalable icons 
to accompany texts with specified and varied fonts in order to easily associate an icon 
to a particular sentence, as noted by Baker (Column 5, lines 40-55). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Baker's would have allowed Hyon's to provide a method for allowing for scalable icons 
to accompany texts with specified and varied fonts in order to easily associate an icon 
to a particular sentence, as noted by Baker (Column 5, lines 40-55). 

Hyon and Baker do not explicitly teach: 
E) said text attribute data containing color attribute data indicating at least a fore color 
of a corresponding predetermined kind of character : 

H) converting a color of said registered image according to said color attribute data . 

Ostermann, however, teaches " said text attribute data containing color 
attribute data including at least a fore color of a corresponding predetermined 
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kind of character " as "The increased intensity of the emoticon may be accomplished 
by changing the icon from black-on-white background to black-on-colored background 
(such as red or yellow) where the intensity of the background color reflects the 
amplitude" (Column 9, lines 43-67) and "The sender can associate typed words with an 
emoticon by underlining, coloring, highlighting, or by any other means. For example, 
the method may comprise providing the sender an option to assign a color to the at 
least typed one word such that the chosen emoticon begins to be presented by the 
animated entity to the recipient at the first typed word with the assigned color and the 
chosen emoticon presentation by the animated entity ends at the last typed word with 
the assigned color" (Column 1 1 , lines 43-50) and " said display control portion 
converts a color of said registered image according to said color attribute data " 
as "The sender can associate typed words with an emoticon by underlining, coloring, 
highlighting, or by any other means. For example, the method may comprise providing 
the sender an option to assign a color to the at least typed one word such that the 
chosen emoticon begins to be presented by the animated entity to the recipient at the 
first typed word with the assigned color and the chosen emoticon presentation by the 
animated entity ends at the last typed word with the assigned color" (Column 1 1 , lines 
43-50). 

It would have been obvious to one of ordinary skill in the art at the time the 
invention was made to combine the teachings of the cited references because teaching 
Ostermann's would have allowed Hyon's and Baker's to provide a method to allow 
the ability to insert a customized emoticon or specified image by a single button click, as 
noted by Ostermann (Column 3, lines 35-54). 

Response to Arguments 
7. Applicant's arguments filed on 04/05/2010 have been fully considered but they 
are not persuasive. 

Applicants argue on page 13 that "these icons are also part of the display of 
the operating system and are not images corresponding to image codes in a 
correlated manner according to registration processing by a user". However, the 
examiner wishes to state that the primary art of Hyon teaches the aforementioned. 
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Specifically, the examiner wishes to refer to Paragraphs 23, 25, and 44 of Hyon which 
state "A storage 18 is comprised of a ROM (Read Only Memory) and a RAM (Random 
Access Memory) for storing programs and data, and a voice memory. The storage 18 
stores an operation program for inputting emoticons and a plurality of emoticons in the 
form of a bit map according to the embodiment of the present invention. The plurality of 
emoticons are stored by groups in the storage 18 to facilitate selection of emoticons" 
(Paragraph 23), "storage 18 stores oriental or occidental emoticons according to the 
cultural area of a user" (Paragraph 25), and "A plurality of emoticons are pre-stored in a 
mobile terminal so that a user easily select an intended emoticon in the present 
invention. In another embodiment, the user can add hieroglyphics to basic emoticons 
stored by a manufacturer. The hieroglyphics can be stored in three ways: the user 
directly stores them, receives them from a base station and the stores them, or 
downloads them from the Internet. The user can change and edit the emoticons stored 
in the mobile terminal" (Paragraph 44). The examiner further notes that it is clear that 
the emoticons which are edited, changed, and user-generated must be stored in the 
RAM of storage 18 since items stored in RAM can be written to whereas items stored in 
ROM cannot be written (i.e. are Read-Only). The examiner further notes that it is clear 
that a user "registers" the emoticons in Hyon because the user is allowed to 
modify/edit/create emoticons. 

Applicants argue on page 13 that "nor are these icons intended to be inserted 
into a textual sentence in the manner described by the claimed invention". 
However, the examiner wishes to refer to Column 8 of Ostermann which states "The 
sender may also insert emoticons 103 into the text of the message. The system 
includes predefined emoticons 96, such as ":-) " for a smile, "::-)" for a head nod, "*w*" 
for an eye wink, and so forth. The predefined emoticons are represented either as icons 
or as text, such as ";-)". As shown in FIG. 6, the window 80 includes a sample group of 
emoticon icons 96. The sender inserts an emoticon into a text message at the location 
of the cursor 102 by clicking on one of the emoticon icons 100. The sender may also 
type in the desired emoticon as text. Emoticon icons 96 save the sender from needing 
to type three keys, such as ":" and "-" and ")" for a smile. The icons 96 may be either a 
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picture of, say, a winking eye or a icon representation of the characters ";-)" 100, or 
other information indicating to the sender that clicking on that emoticon icon will insert 
the associated emotion 103 into the text at the location of the cursor 102" (Column 8, 
lines 35-50). The examiner further wishes to state that it is clear that that emoticon is in 
between words of a sentence. 

Applicants argue on page 13 that "Ostermann merely recites text having a 
particular color is assigned to an emoticon. The reference is silent regarding 
converting a color of a registered image according to color attribute data in the 
text attribute data of the text in the sentence with the registered image". However, 
the examiner wishes to refer to Columns 9 and 1 1 of Ostermann which state "The 
increased intensity of the emoticon may be accomplished by changing the icon from 
black-on-white background to black-on-colored background (such as red or yellow) 
where the intensity of the background color reflects the amplitude" (Column 9, lines 43- 
67) and "The sender can associate typed words with an emoticon by underlining, 
coloring, highlighting, or by any other means. For example, the method may comprise 
providing the sender an option to assign a color to the at least typed one word such that 
the chosen emoticon begins to be presented by the animated entity to the recipient at 
the first typed word with the assigned color and the chosen emoticon presentation by 
the animated entity ends at the last typed word with the assigned color" (Column 1 1 , 
lines 43-50). The examiner further wishes to state that assigning a color to a word 
clearly teaches the claimed color attribute data of text. Moreover because the color of 
the emoticon is the same color as that of the assigned text color, it is clearly converted 
because it matches the desired color. 

Applicants argue on pages 13-14 that "none of Hyon, Baker, or Ostermann 
teaches or suggests a display control portion... Baker fails to disclose inserting a 
registered image into a sentence with text, and merely describes scaling an 
operating system icon to the font size of text used within the window". However, 
the examiner wishes to state that the cited art of Ostermann teaches the claimed 
insertion into a sentence. Moreover, the cited art of Baker clearly teaches scaling 
emoticons to match font sizes of text characters. Therefore, the combination of Baker 
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with Ostermann (along with the primary reference of Hyon) clearly teaches the 
aforementioned. 

Conclusion 

8. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

U.S. Patent 6,987,991 by Nelson on 17 January 2006. The subject matter 
disclosed therein is pertinent to that of claims 1, 4-17, and 19 (e.g., methods 
automatically scale graphics into a text sentence that is sent to another user). 

U.S. PGPUB 2002/0120653 by Kraft et al. on 29 August 2002. The subject 
matter disclosed therein is pertinent to that of claims 1, 4-17, and 19 (e.g., methods 
automatically scale graphics into a text sentence that is sent to another user). 

U.S. Patent 6,584,479 by Chang et al. on 24 June 2003. The subject matter 
disclosed therein is pertinent to that of claims 1,4-17, and 19 (e.g., methods 
automatically scale graphics into a text sentence that is sent to another user). 

U.S. Patent 6,456,305 by Qureshi et al. on 24 September 2002. The subject 
matter disclosed therein is pertinent to that of claims 1 , 4-1 7, and 1 9 (e.g., methods 
automatically scale graphics into a text sentence that is sent to another user). 

U.S. Patent 5,301 ,1 06 by Hersum on 05 April 1 994. The subject matter 
disclosed therein is pertinent to that of claims 1 , 4-1 7, and 1 9 (e.g., methods 
automatically scale graphics into a text sentence that is sent to another user). 

Contact Information 

9. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Mahesh Dwivedi whose telephone number is (571) 272- 
2731 . The examiner can normally be reached on Monday to Friday 8:20 am - 4:40 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Tim Vo can be reached (571 ) 272-3642. The fax number for the 
organization where this application or proceeding is assigned is (571) 273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
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Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov . Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 

Mahesh Dwivedi 
Patent Examiner 
Art Unit 2168 

May 04, 2010 
/Mahesh H Dwivedi/ 
Examiner, Art Unit 2168 



